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ABSTRACT 

This evaluation of the California Title-Author List 
(CATALIST), second edition, a union catalog on microfiche published 
from computer output microform tapes which contains location records 
for approximately 700,000 titles held in 152 public, academic, and 
special libraries throughout California, is presented as a means of 
assessing progress to date and providing information for future 
planning for three interrelated statewide resource sharing projects 
involved in the creation and distribution of CATALIST: the California 
Database-Monographs Project, the Statewide Data Base Program, and the 
Finding List Project. The evaluation effort includes an analysis of 
the contents of CATALIST, a general survey of all 251 CATALIST users 
to obtain usage and effectiveness perceptions, and further collection 
of data from CATALIST users to ascertain the hit rate of all finding 
tools used during the interlibrary loan (ILL) search process. In 
terms of content, CATALIST is determined to be public library 
oriented. It is further shown that CATALIST receives widespread use 
in support of interloan services and that it is effective and easy to 
use. Areas for improvement, notably inclusion of more titles and more 
evenly distributed geographic coverage, and a variety of factors 
influencing the future of CATALIST are identified. The report 
concludes with three appendices: samples of the general survey and 
data collection forms, and a list of comments received on the impact 
of CATALIST upon ILL patterns of individual libraries or systems. 
(ESR) 
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Evaluation of CATALIST, 2nd Edition 



- "Introduction 

During the Fall of 1981 and the early part of 1982 the California State 
Library conducted an evaluation of the 2nd Edition of the microfiche finding 
list, CATALIST, as means of assessing the progress of several statewide library 
resource sharing projects, notably the Library Services and Construction Act 
(LSCA)^funded California Database - Monographs project, the California Library 
Services Act (CLSA) Statewide Database program, and the LSCA funded Finding 
List project. This report presents the results of the evaluation effort. 

The evaluation was divided into three major areas of investigation: 

(1) An analysis of the contents of the 2nd Edition to determine 
the scope and comprehensiveness of the information contained in 
the finding tool . 

(2) A general survey of all known purchasers of CATALIST, 2nd 
Edition to determine the amount and type of use, the per- 
ceived effectiveness of the tool, and the degree to which 
potential problems attendent on the construction of large, 
batched, bibliographic databases affected the usefulness of 
the finding list. 

(3) A hit rate sample to provide, a quantitative measure of the 
performance of the finding tool in comparison to other 
location finding methods and tools available in'tKe slate. 
In addition statistical data on public libraries available 
at the State Library was also analyzed and correlated with 
the results of the three major lines of inquiry. 

It should be noted that all of the data^and findings presented in this 
report fefer, unless otherwise noted, to the 2nd Edition of CATALIST (the 3rd 
Edition was published in March, 1982) and/or to the period October to December 
1981 (the time period of the general survey and hit rate sample). 

Acknowledgement is made to Karl Pearson and Catherine Fine of the Cali- 
fornia Library Authority for Systems and Services (CLASS) for their invaluable 
assistance in much of the work that forms the basis of the contents analysis 
section; to the several members of the California Library Services Board who 
aided in the development of the survey instruments; to Katherine Page, then 
of the 49-99 Cooperative Library System for her help .and-advice; to Linda 
Downing of California State University, Sacramento for her assistance and 
advice on the use of the Statistical Program for the Social Sciences (SPSS); 
and especially to the many librarians who gave so much time and thought to 
completing the surveys. 
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Evaluation of CATALIST, 2nd Edition 



Executive Summary 

Purpose of the evaluation : 

During the Fall of 1982 and the Winter of 1982 the California State Library 
conducted an evaluation of the microfiche finding tool, CATALIST, then in its 
2nd Edition, as a means of assessi.ng progress to date, and providing information 
useful to future planning, for three interrelated statewide projects: (1) the 
LSCA-funded California Database -Monographs Project; 2) the CLSA-funded State- 
wide Data Base Program; and 3) the LSCA-funded Finding List Project. 

Objectives of the Evaluation Effort : 

1) To determine the scope and coverage of the 2nd Edition as a means of 
assessing progress in the building of the database, 

2) To determine the amount and types of use of the 2nd Edition in libraries^ 

3) To determine the perceived effectiveness and value of the 2nd Edition 
in support of various library activities. ^ 

4) To determine if any significant problems, as p"ferceived by CATALIST users, 
had detracted from the use or effectiveness of the finding tool. 

Methodology : 

The evaluation effort was divided into three main elements: 1) An analysis 
of the contents of the 2nd Edition, including random samples 'of actual listings; 
2) A general survey of all CATALIST users to obtain usage and effectiveness per- 
ceptions; and 3) A survey of CATALIST users to determine the hit rate of all 
finding tools then in use in their individual libraries. 

Results & Findings : 

- Both in terms of content (almost 98%) and use (over 98X) CATALIST, 2nd 
Edition is a public library oriented finding tool. 

- Compared to an estimate of the total public library holdings in the 
state, the 2nd Edition contained over 12% at the time of publication, 

- Individual public library systems and individual libraries varied widely 
in representation in the finding tool. 

- Titles were well distributed by publication date. 

- Recent (1979/80 imprints) titles had considerably more (17.9) locations 
per title than the average (3.1) for the publication as a whole. 

- CATALIST, 2nd Edition was used at a rate of over 96,000 uses annually. 

- Over 86% of 2nd Edition usage was in support of interloan activities. 

- On a scale* -of -Hto 10 the 2nd Edition received a 7.4 effectiveness 
rating in support of j'nterlibrary loan. 

- On a scale of T to 10 the most significant (4.6) problem identified by 
users was lack of titles. - ^ 

- The 2nd Edition achieved an overall hit rate of 61%, ranking it third 
among the finding tool categories measured. 

- Local libraries exhibit a (marked preference for searching local or 
regional finding tools first in attempting to locate requested items, even 
though such tools are not as comprehensive as national or statewide catalogs, 
probably because local communication and delivery arrangements are faster and/or 
cheaper. 



- The^2nd Edition recorded a 29% local (delivery area) hit rate during 
the sample period - higher than any other national or statewide finding tool 
included in the survey • 

Conclusion : 

The 2nd Edition of CATALIST received widespread use in support of interloan 
services and was found to be effective and easy to use. Areas for improvement 
were identified, notably in regard to greater comprehensiveness (i*e», more titles) 
and more evenly distributed geographic coverage. Many factors will influence the 
future of CATALIST, including the capability of microform technology to provide 
practical access to large files, the extent to which local library jurisdictions 
are able to implement new technology that may provide more rapid and cost-efficient 
interlibrary loan access to their collections, and the availability of funding to 
support cooperative library resource-sharing efforts. Whatever the future may 
hold, the 2nd Edition of CATALIST provided valuable and effective support for co- 
operative lending services in California's public libraries and will serve as an 
invaluable benchmark against which future plans and activities can be measured. 



Evaluation of CATALIST, 2nd Edition 



Background 

CATALIST (California Title-Author List) was first published in a Pilot 
Edition in the Fall of 1979. Following the resolution of technical processing 
problems revealed in the Pilot Edition the first full edition was published in 
the Spring of 1980. The 2nd Edition, which is the subject of this evaluation, 
was published in.March of 1981. The 3rd Edition was published in March of 1982. 

CATALIST is the final output product of a batched offline merge of machine- 
readable bibliographic records drawn from a variety of sources: (1) Under the 
state funded California Statewide Database program (a component program of the 
California Library Services Act) bibliographic records of participating public 
libraries throughout the state are collected in the form of magnetic tape copies 
of their current (1978+) acquisitions. These machine readable records are 
obtained from two major sources - a) archival tapes representing local public 
library cataloging done by means of the two major online bibliographic utili- 
ties in widespread use in California (OCLC (the Online Computer Library Center) 
and.RLIN (the Research Libraries Information Network)); and b) magnetic tapes 
of the Marc records of books ordered from two major book vendors (Baker & 
Taylor and Brodart) by participating public libraries who are not currently 
using the cataloging utilities. (2) Under the federally funded (LSCA) Cali- 
fornia Data Base - Monographs (CDB-M) project the machine-readable retro- 
spective records of a) several public libraries, and b) a few academic and 
special libraries, and c) a portion of the California Union Catalog have been 
added to the database and published in CATALIST. 

These records represent the results of a wide variety of retrospective 
conversion techniques including both online and offline methods and are 
selected annually for inclusion on the basis of the availability of funds to 
pay for processing, the desirability of the files themselves, and the availi- 
bility of machine-readable records in a format reasonably compatible with that 
of the masterfile. 

Once records have been collected they are matched against MARC files (if 
not already in MARC format), duplicates eliminated, and then they are merged 
with the masterfile. 

Neither the CLSA Statewide Data Base Program nor the LSCA CDB-M project 
make any provision for access to the information contained in the database; 
the two projects are solely concerned with building and maintaining the files. 
Since the publication of the 1st edition of CATALIST, funds to provide access 
for participating public and state agency libraries have come from the LSCA 
funded Finding List Project. This latter project provides reimbursement to 
qualifying libraries which purchase a full copy of CATALIST. In its initial 
year the Finding List Project also provided funds for the purchase of CULP 
(California Union List of Periodicals) and in its third and final year 
(1982-83) win provide only partial reimbursement for the purchase of CATALIST. 
The chart on page 5 illustrates the relationships between the three separate 
funding sources .and the many data sources that contribute to building the data- 
base and publishing CATALIST. 
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The annual match/merge process is designed to upgrade the maximum possible 
number of records to full MARC format, remove ajl identifiable duplicate biblio- 
graphic records, reformat_£he records to CDB-M masferfile internal format and 
merge all of the records into one single file. This file (the "new" CDB~M 
masterfile) then 'becomes the source for the COM (Computer Output Microform) 
tape from which an edition of CATALIST is published. 

The California Library Authority for Systems and Services (CLASS), under 
contract for the LSCA funded CDB-M Project and the CLSA Statewide Database 
Program, provides database management services for the masterfile building, 
maintenance, and publication effort. Data processing services are furnished, 
by contract with CLASS, by AutoGraphics , Inc. of Monterey Park. 
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Evaluation of CATALIST, 2nd Edition 



Contents Analysis 



The 2nd Edition of CATALIST was published in March of 1981 and contained 
an estimated 2,200,000 title locator records^ representing about 700,000 
titles held in 152 libraries throughout the state. Of these 141 are public 
libraries and the remaining eleven are academic and special libraries- The 
eleven non-public libraries account for less than 2% of the holdings listed 
in the finding tool. Thus it is fair to say that CATALIST, 2nd Edition is 
primarily composed of bibliographic and holdings information pertaining to 
public library collections in California. 

For the most part California's 168 public libraries are organized in 15 
cooperative library systems composed of from 3 to 28 geographically contiguous 
public library jurisdictions. This geographic division of the state provi.des 
one way to view the contents of CATALIST, 2nd Edition, The chart on page 8 
shows the geographic distribution by public library cooperative system of the 
title locator recprds in the 2nd Edition. This distribution of title locations 
reflects the operation of several variables, the most significant of which are: 
(1) the degree to which individual libraries and library systems have converted 
their catalogs to machine-readable form; (2) the availability of LSCA funding 
to add retrospecti ve records to the masterfile; (3) the number and size of the 
libraries comprising each system; (4) the number and size of libraries utilizing 
one of the two bibliographic utilities (RLIN/OCLC) and/or one of the two book 
vendors (Baker & Taylor/BroDart) from which data for the CLSA Statewide Database 
Project is collected; and (5) the number and size of libraries contributing to 
the California Union Catalog, a portion of which appears in CATALIST, 2nd Editijon. 

An additional perspective on the distribution of title locators in the 
2nd Edition may be gained by comparing the data displayed in the chart on page 
with an estimate of what the result would be if alj^ of the holdings of all of 
California's 168 public libraries were converted to machine readable form and 
combined in a single database. Since many public libraries do not routinely 
account for their collections by the number of titles held the chart of total 
title locator records as of 1981 (page 9) is an estimate based on statistics 
derived from the 1982 edition of the California Library Statistics and Directory . 

Where actual data on the number of titles in an individual reporting 
library's collection as of 1981 was available that data was used. Where data 
was unavailable, the average number of titles for all libraries of similar 
size (measured by population served) was used. The chart at the bottom, of ' 
pagel3 displays the average number of titles for each library size group. 

A comparison of the two charts reveals that the distribution of title 
locator records in CATALIST, 2nd Edition is significantly different from that 
of the estimated total statewide title locator file. For example, the Bay 
Area Library Information System (BALIS) share of the 2nd Edition is 18*3^ 
while BALIS' share of the estimated total locators in the state is substan- 
tially lower, 8%. This reflects the fact that a major retrospective conver- 
sion effort involving three BALIS members resulted in a file which was added 
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A title locator record is defined as one library location appended to the 
bibliographic record pertaining to a title in that library *s collection. 
Thus one title (or bibliographic record) may have many title locators (or 
O holdings statements) attached. 
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CATALIST. 2nd Edition, March 1981 
Representation by System (Title Locators) 



BLACK GOLD 
326,077 
14.8% 




2,200,000 TITLE LOCATORS TOTAL 
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Total Title J.ocator Records in California 
Public Libraries by System 



Other BALIS 




Total Title Locator Records = 18,062,654 
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to the CDB-M for the 2nd Edition of CATALIST. In contrast, Santiago Library 
System, roughly comparable to BALIS in terms of the number of total title 
locator records available in the system (again about 8^ of the statewide total) 
has only a 2.8% representation in the 2nd Edition — this reflects the fact 
that no major retrospective files of Santiago member libraries had been added 
to the database at the time the 2nd Edition was published. 

It should be noted that even though BALIS is, in the above sense, over- 
represented in the 2nd Edition, the BALIS holdings listed in the finding tool 
are still only a little over 2.8% of the potential number of title locators 
for that system. The chart on page 11 displays, by system, the number of 
title locators in the 2nd Edition, system representation in the 2nd Edition 
as a percent, the estimated total number of title locators for the system, 
the percentage of the system's estimated total share of a complete statewide 
file, and the percentage of 2nd Edition records as compared with the estimated 
system total (i.e., for the latter, the number of published 2nd Edition records 
divided by the estimated total number of records for the system). Of an esti- 
mated 18,062,554 public library title location records (as of 1981) , 2,200,000 
or about 12.2% have been converted to machine-readable form and published in 
CATALIST, 2nd Edition. 

Of equal interest is the distribution of title locator records by indivi- 
dual library. The chart on page 12 displays the number of title locator records 
for the 26 libraries with over 25,000 locator records in the 2nd Edition. These 
26 libraries* records constitute 69.2% of the total holdings listed in the 2nd 
Edition. As with the distribution of title locator records by public library 
system the data illustrates the effect of retrospective conversion and the 
availability of LSCA funds to add retrospective records to the database through 
the CDB-M project as well as the expected variations due to the size of the 
individual libraries. While many of the libraries listed are among the largest 
in the state, some are considerably smaller than their postion on the list might 
indicate. The chart on page displays the relative size (based on population 
served) of public libraries in California and their representation (in terms of 
number of title locators) in CATALIST, 2nd Edition. The accompanying chart dis- 
plays the averag e size, in terms of number of titles held, of public library 
collections of tKe six groups. 

2 

A random sampling of 526 title records found in the 2nd Edition was taken 
by staff at the California Library Authority for Systems and Services (CLASS). 
The distribution of the sample by publication date is displayed in the chart on 
page . The sample data support the conclusion that CATALIST, 2nd Edition 
contains a large number of retrospective bibliographic records. Nearly 52% of 
the titles sampled represented Dre-1971 oubjication dates. Of the estimated 
338,100 titles with publication dates of 1971-1980 about 33,000 represent titles 
with 1979 or 1980 imprint dates. Staff at CLASS randomly sampled 526 titles of 
this segment (1979/1980 imprints) of CATALIST, 2nd Edition. The chart on page 
displays the number of title locator records per title record in this subset of 
CATALIST records. 



Note that a title record is not the same as a title locator record (see footnote 
1, page ). A title record is the bibliographic information pertaining to a 
unique monograph and may have many locator records attached to it. 
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Title Locator Records in CATALIST, 2ncl Edition 



and 



Total Title Locations in CaliOornia Public Libraries by System 



System 



BALIS 

Black Gold 

49-99 

Inland 

MCLS 

MOBAC 

MVLS 

North Bay 
North State 
Peninsula 
SJVLS 
Santiago 
Serra 
South Bay 
South State 
Other 



A 

Title Locators 
in 2nd Edition 



403,589 
326,077 
86,337 
88,178 
304,009 
28,266 
211,044 
268,425 
64,820 
37,534 
75,809 
61,777 
90,893 
62,116 
61,703 
29,423 



B 

% age of Total 
2nd Edition 



Total Title j 
Locators in System 



18.3 


1,742,713 


14.8 


680,071 


3.9 


747,307 


4.0 


1,308,608 


13.8 


2,867,567 


1.3 


636,728 


9.6 


894,029 


12.2 


1,215,976 


3.0 


896,737 


1.7 


982,654 
756,511 


3.5 


2.8 


1,294,438 


4.1 


1,373,376 


2.8 


1,101,701 


2.8 


610,876 


1.4 


953,362 



D 

7. age of Total Public 
Library Title Locators 



9.7 
3.8 
4.1 
7.2 
15.9 
3.5 
4.9 
6.7 
5.0 
5.4 
4.2 
7.2 
7.6 
6.1 
3.4 
5.3 



% age of Total System Title 
Locators in 2nd Edition (A-fC) 



23.2 
47.9 
11.6 
6.7 
10.6 
4.4 
23.6 
22.1 
7.2 
3.8 
10.0 
4.8 
6.6 
5.6 
10.1 
3.1 



Total 



2,200,000 



100.0 



18,062,654 



100.0 



12.2 



1. Estimated using a combination of actual reported statistics and, for libraries not reporting, derived averages from 
data in the California Library Statistics and Director y 1982 (1981 reports). 
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Title Locators in CATALIST, 2nd Edition 
Single Library Representation 



Percent of Cumulative 
Total Percent 



Libraries with over 100,000 title locators in CATALIST, 2Rd Edition 
Group I Alameda County 114,115 1 

(3 librar- Contra Costa County 109,242 > 400,627 or 18. 25^ 18.2 

ies) Sacramento City-County 177,270 J 



Libraries with 75,000 - 100,000 



Group II 
(2 librar- 
ies) 



Santa Barbara 
Los Angeles P.L, 



94,658 
80,075 



174,733 or 7.9% 



26.1 



Libraries with 50,000 - 75,000 



Group III 
(7 librar- 
ies) 



San Luis Obispo 
Santa Maria 
Pomona 

Marin County 
Solano County 
Sonoma County 
San Francisco 



64,420 
54,457 
71 ,211 
71 ,096 
56,842 
58,969 
62,787 j 



> 439,782 or 20.0% 



46.1 



Libraries with 25,000 - 50,000 



Group IV 
(14 librar- 
ies) 



Group V 
(126 

libraries) 



Stanislaus County 


38,421 


Stockton-San Joaquin 


29,202 


Alameda Public 


48,749 


Richmond 


26,382 


Lompoc 


44,625 


Ventura County 


45,017 


Riverside 


25,572 


Pasadena 


45,131 


Napa 


39,226 


San Mateo County 


25,283 


Fresno 


35,775 


San Diego P.L. 


34,382 


Santa Clara County 


32,288 


Los Angeles County 


38,499 


than 25,000 




All others 





} 508,552 or 23. If; 



69.2 



676,306 or 30. 8r; 



100.0 
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Representation in CATALIST, 2nd Edition, by Public Library Size (Population Served) 



Titles/Locators 
in 

\^ATALIST 
Size of Lib.^^"^,^^ 
(Pop'< ..Serv.) 


Group A 

Over 
100,000 

Title 
Locators 


Group B 
75,000- 
100,000 
Title 
Locators 


Group C 
50,000- 
75,000 
Title 
Locators 


Group D 
25,000- 
50,000 
Title 
Locators . 


Group E 
Less than 
25,000 
Title 
Locators 


Total 
in 

CATALIST 


Total 

in 
State' 


Group 1 
Over 500^,000 
Population Served 


2 


1 


1 


4 


3 


11 


12 


Group 2 
100,000-500,000 
Population Served 


1 


4 


5 


19 


30 


32 


Group 3 
50,000-100,000 
Population Served 


i 

: 

1 

; -0- 

i 


.0- 


2 


4 


30 


36 


41 


Gr6up 4 
25,000-50,000 
Population Served 


-0- 


! -0- 


I 

-0- 


1 


27 


! 28 


32 


Group 5 
12,500-25,000 
Population Served 


-0- 


! 

-0- 


-0- 


-0- 


'. 23 


23 


28 


Group 6 
Less than 12,500 
Population* Served 


-0- 


-0- 


-0- 


-0- 


i 

I 13 


13 


J . 

\ 

23 


Total 
in 

CATALIST 


3 


2 


7 


14 


115 


141 


168 



Average Number of Titles/Volumes by Public Library Size 



Group 


Average 11 of titles 


Average # of volumes 


volumes/title ratio 


1 


259,365 (9) 


1 ,698,683 (12) 


6.549 


2 


169,483 (24) 


407,578 (30) 


2.405 


3 


111,119 (29) 


173,325 (40) 


1.560 


4 


82,423 (14) 


99,236 (30) 


1 .204 


5 


52,440 (17) 


61,637 (27) 


1.175 


6 


29,323 (14) 


34,336 (21) 


1.171 



( ) = Number of Libraries Reporting 
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CATALIST. 2nd Edition 
Distribution of Titles by Date of Publication* 



Date of 
Publication 


Pre - 
1901 


1901- 
1910 


1911- 
1920 


1921- 
1930 


1931- 
1940 


1941- 
1950 


1951- 
1960 


1961- 
1970 


1971- 
1980 


Total 


§ of Titles 
in Sample 


9 


3 


9 




13 


28 


51 


148 


254 


526 


Estimated # 
of Titles in 
2nd Edition 


11,900 


4,200 


11,900 


14,700 


17,500 


37,100 


67,900 


196,700 


338,100 


700,000 



* Based on random sample taken by CLASS staff. 



Based on this sample the average number of locations per title in the 
1979/80 portion of CATALIST, 2nd Edition is 17.9. A relatively small number 
of these recent titles, 3,500 or 10.8%, are held by only one library. These 
figures contrast sharply with the same measures for the 2nd Edition as a whole 
(all imprint dates) - the average number of locations per title for all titles 
in the 2nd Edition is 3.1; and the number of titles with only one location for 
the entire 2nd Edition is 336,750 (48% of the titles listed). This high pro- 
portion of single locator titles in the pre-1978 portion of the file as com- 
pared with the post-1978 portion (nearly 50% versus about 11%) reflects the 
operation of the five variables mentioned in the beginning of this section 
(see page ) as well as the common sense observation that the older a title 
is the less probable it is that many libraries would choose to replace it if 
it is lost or damaged. 



CATALIST, 2nd Edition 
Distribution by Number of Locations per Title (1979-1980 Imprint Only)* 



Number of 
Locations 


1 


2-3 


4-6 


7-10 


11-15 


16-20 


21-30 


31-40 


41-50 


more 
than 
51 


Total 


Number of 
Titles 


57 


53 


59 


64 


47 


53 


89 


54 


26 


24 


526 



* Based on a random sample taken by CLASS staff. 
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Summary 



The 2nd Edition of CATALIST contained about 2,200,000 title locator 
records representing the partial holdings of 141 of California's public 
libraries. Based on an estimation of 18,062,654 title location records for 
the holdings of all 168 California public libraries^ in 1981, CATALIST con- 
tained listings Tor 12.2%, The geographical distribution of the title 
location records in CATALIST, 2nd Edition is, in many instances, at consider- 
able variance with the geographical distribution of the estimated total public 
library title location records. For four systems (BALIS, Black Gold, MVLS, 
and North Bay) proportional reoresentation in CATALIST, 2nd Edition is 1.8 
(North Bay) to 4.2 (Black Gold) times those systems' expected portion of a 
complete state public library file. This difference is the result of the 
operation of several variables, the most significant of which are the size 
of the libraries contributing records to the data base, the extent to which 
individual 'libraries and systems have converted their records to machine- 
readable form, and the availability of funds to pay the cost of processing 
records into the catabase. As the database continues to grow through the 
combined operation of the LSCA CDB-M Project and the CLSA Statewide Database 
Program; and, as more retrospective conversion projects are completed by 
individual libraries and systems, it can be expected that the geographic 
distribution of title location records in the finding tool will more closely 
approximate the geographic distribution of public library resources in the 
state. 

The distribution of titles in the 2nd Edition by date of publication 
indicates that CATALIST is a relatively well-balanced finding tool (i*e, 
contains records representing publications in all decades of the 20th 
century in proportions that smoothly diminish with age). An exception to 
this conclusion is the most recently published material. 1979/1980 materials 
account for only 4.7% of the database as a whole and 9.8^ of the 1971-1980 
portion of the file. This is probably the result of the time needed to 
collect data from many different libraries, systems, and vendors; batch 
process this data; and publish it. 
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"Note that this estimate includes both machine-readable (converted) and 
unconverted records. 
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General Survey Results 



The second major segment of the CATALIST, 2nd Edition evaluation effort 
was a general survey of all purchasers and users of the finding tool. Indivi- 
dual libraries acquired the Second Edition either by purchasing the tool with 
local funds or by participation in the LSCA funded Finding List Project. That project 
provided funds not only for the microfiche set itself but, for those libraries 
who did not already possess compatible microfiche readers, the equipment neces- 
sary to use the finding tool. In all 251 libraries were surveyed. The chart on 
page 17 displays the response rate by library type, for responding public libraries 
by size, and by participation in the Finding List Project. 

By far the largest number of responses and the highest response rate was 
that for individual' public libraries, which accounted for 133 or 76.9% of the 
total responses received. If public library systems responses are combined with 
the individual library responses, public libraries account for 81.5% of the com- 
pleted surveys. 

The survey instrument was designed to elicit information on the usage and 
preceived effectiveness of the 2nd Edition. A complete copy of the survey 
instrument with cumulated responses may be found as Appendix A of this evaluation 
report. Please note that many of the cumulated or aggregate responses shown in 
Appendix A are averages or weighted averages of all responses received. Since 
not all libraries answered all of the questions, the number of libraries responding 
to each question is also shown. The individual responses to the survey were con- 
verted to machine-readable form and tabulated at the California State University, 
Sacramento Computer Center where the availability of SPSS (the Statistical Pro- 
gram for the Social Sciences) allowed the data to be manipulated and cross tabu- 
lated in a variety of ways. 



Usage of CATALIST, 2nd Edition 

Questions 3, 4, and 5 of the survey were designed to measure the amount and 
type of CATALIST usage. The responses to question 4, "Approximately how often 
per month is CATALIST, 2nd Edition, used in your library?", are displayed in the 
chart on page 18 along with an estimation of the total annual usage based on the 
responses. The data indicates an average usage of 47 times per month per library, 
including the 21 libraries who reported no usage at all. It should be noted that 
the most frequently cited reason for non-usage was that all interlibrary loan re- 
quests were forwarded by the reporting library to a centralized ILL center for 
verification and searching. The reported overall usage supports a conservative 
estimate of over 96,000 uses annually. 

Although CATALIST was designed primarily as a location finding tool , any 
such large aggregation of bibliographic data has several other possible library 
uses Question 5 of the survey was designed to measure the approximate amount, 
expressed as a percentage, of CATALIST usage in support of several library func- 
tions. The chart on page 19 shows the responses to this question. The first two 
uses listed, obtaining ILL holdings and bibliographic verification, account for 
over 74% of the total usage and the next highest reported usage is closely allied - 
obtaining holdings data for direct loans.' These three uses account for nearly 



^A direct loan occurs when a patron of one library jurisdiction borrows a book 
from a neighboring library directly without utilizing ILL services. 
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Response to CATALIST, 2nd Edition Survey 





Type of Library 


Number 
Surveyed 


Responses 


Response 
Rate by type 


Percentage of 
Total Response 


1 

1 • 


ruu lie L 1 Ui di y 






04 . / /o 




0 
L • 


rUDiic Li Drary oystem 


1 ^ 

\ 0 


o 
0 




4 • 0 /0 


0 

%3 • 


o ta tc Myency l i urary 


\ c 


7 


DO . o/o 




A 

H • 


V/Orrec 1 1 Una 1 inotitutiun 


CO 


in 


40 . 0/0 


□ • 0/& 


C 
D . 


nLauciHiL Liurary 




in 


D\J • U.o 


D tOfO 


6. 


state Hospital 


4 


2 

Kb 




1 .2% 


7. 


Special 


20 


3 


15.0% ^ 


1.7'^ 


Totals 


251 


173 


68.9% 


100. Of; 



Distribution of Responding Public Libraries by Size (Population Served) 





Over 
500,000 


100,000 
to 

500,000 


50,000 

to 
100,000 


25,000 

to 
50,000 


12,500 

to 
25,000 


Under 
12,500 


Total 


r of libraries 
responding 


7 


28 


38 


30 


15 


14 


133 


Percentage of 
total P.L. 
Responses 


5 • 3^/0 


21. OS 


28.6?; 


22.6% 


12.0!i: 


10. 5f. 


ioo?i: 



Finding List Participants/Purchasers 





Finding List Participant 


Purchaser 


Total 


of libraries responding 


160 


13 


173 


Percentage of total response 


92.5?; 


7.5% 


loo:. 
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Frequency of CATALIST Use 





A 


B 


C 




Number of 
Libraries Reporting 


Estimated (Mid-range) 
Uses/Month/Library 


Estimated 

Annual Usesl 


Did not answer 
question 


5 


0 


1 

0 ; 


i 0 times/month 


16 


0 


0 , 


! Less than 5 
times/month 


13 


2.5 


390 i 

S 


1 6-10 times/ 
; month 


1 

21 


8.0 


2,016 


11-25 times/ 
month 


1 

i 33 


18.0 


; 7,128 

1 


26-50 times/ 
1 month 


35 


38.0 


I 

15,960 


1 51-100 times/ 

1 month 
* 


27 


75.5 


24,462 


101-200 times/ 
month 


14 


150.5 


1 25,284 

• • 


More than 200 
times/month 


9 


200.0 + 


! 21,600+ 

i 


TOTAL 


173 

1 


^^^^^^^^^^^;><^^ 96,840 + 



^ Column A x Column 8x12 = Column C 

• CATALIST is used in excess of 96,000 times per year. 

• ]2% (21 libraries) of the responding libraries are not using CATALIST at all. 



The most frequently cited reason for non-usage was that all interlibrary loan 
requests were forwarded to a centralized ILL service for verification and searching. 
A few libraries reported that staffing cutbacks were so severe that no interloan services, 
were provided during the survey period. 
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Usage of CATALIST for Specific Purposes 



Purpose 


Number of librar- 
ies Reporting 
Some Usagel 


Mean % of 
Reported Use^ 


Mode % of 
Reported Use^ 


Adjusted Percent-, 
age of Total Use* 


Obtainina ILL Holdinas 


130 






J wl • O 


Bib. Verification 


100 


31.3 


10.0 


20.4 


General Reference 


59 


11.2 


5.0 


4.3 


Specific Subject 
Reference 


74 


12.3 


5.0 


5.9 


Acqui sitions Deci sions 


13 


5 8 




U.J 


Catal ogi ng Ai d 


25 


21 3 


1 0 




Direct Loan 


50 


31 .7 


5 0 


10 4 


Publi c Use 


18 


6.3 


1.0 


7 


Other 




75.0 


75.0 


0.5 




100% 



^ 152 libraries responded to this question (#5). The average number of different types of 
use reported v/as 3 per library. 

2 Columns will not total lOO'o due to the nature of the data collected. See "Percentage 
of Total Reported Use" (Column 5) to obtain derived percentages of overall reported use. 

3 

While this column represents overall usage of CATALIST it should not be interpreted as 
representing an "average" library's use of the 2nd Edition. Very few libraries reported 
nx)re than 4 uses for the finding tool, and none reported usage in support of all the 
measured categories. The mean % and mode % columns are more revealing of individual 
library praciice. 
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SS% of the total. Relatively few libraries are using CATALIST as a cataloging 
aid or to help in making acquisitions decisions. While the percentage of overall 
usage is not high (10.2%) the number of libraries reporting some usage of CATA- 
LIST for general and specifix subject reference is quite high (133 or 88% of 
those responding to the question). Very little use {.7%) by members of the public 
was reported. 

The pattern of CATALIST usage revealed in the responses to Question 5 is 
reflected in the location of CATALIST, (question 3 of the survey), with copies 
of CATALIST kept at the reference or ILL desk for over 70% of the responding 
libraries. Only 27 libraries (18%) reported placing their copy of CATALIST in 
technical services sections or other clearly non-public service areas. 

Over half of the reported usage of the 2nd Edition was for the express 
purpose of obtaining holdings data for interlibrary loan. Question 9 of the survey 
was designed to obtain information on how CATALIST was used in conjunction with 
all of the other finding tools available in responding libraries. The chart on 
page 21 tabulates the responses to the question. It should be noted that it was 
necessary to categorize some of the finding tools in order to organize the data 
in an intelligible form. Thus the category "Neighboring Library's Catalog" in- 
cludes the Los Angeles Public Library Book Form Catalog, Orange County's Book 
Form Catalog, etc, and the "System Union Catalog" category includes such dis- 
parate products as Black Gold's distributed COM union catalog. North Bay's cen- 
tralized manual union file, and various Round Robin System TWX searches - all 
of which fill the logical function of a system union catalog but which vary con- 
siderably in terms of ease of use, frequency of publication, and comprehensive- 
ness. 

The data supports the conclusion that for at least 35 (25%) of the 138 
libraries who responded to this question CATALIST was the only finding tool in 
use at the time of the survey. The average number of finding tools in use among 
responding libraries was 2,5, CATALIST was the most frequently cited 1st, 2nd, 
and 3rd search order tool, (Note, however, that only libraries who had pur- 
chased CATALIST were surveyed.) 103 of the reporting libraries were using 2 or 
more finding tools - i,a,, had a choice of which finding tool to use first. By 
combining this fact with the conclusion that 35 libraries had only one finding 
tool (CATALIST) available to them it is possible to calculate the percet.tage of 
responding libraries choosijig^ one finding tool or category of finding tools for 
initial searching of ILL requests. The following chart displays the results of 
this calcuVation. 
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1 Finding Tool 
I 




Number of 
Libraries Choosing 
for Initial Search 


% of Libraries 
Choosing for 
Initial Search 


1 - 

CATALIST 




35 


3^.0% 


System Union Catalog 




37 


35.9% 


OCLC 




5 


A. 9% 


RLIN 




1 


].0% 


Neighboring Library's 


Catalog 


12 


11.6% 


Shared Catalog 




12 


11.6% 


NUC 


Catalog) 


1 


]M 


cue (California Union 


0 


0 


Other 




0 


n 


TOTAL 




im 


100. os; 
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Search Order Usage of Finding Tools by Responding Libraries 





Nun 


iber of nbr< 


Ties report' 


ng usage for 


^ ILL search' 


ng 


1 






oro 


H un 


btn 


1 0 Ud 1 




in 
/u 


4h 




A 

H 


o 
C 




System Union Catalog 


37 


17 


6 


3 


none 
reported 


63 


OCLC 


5 


14 


10 


7 


4 


40 


Dt TM 

KLIN 


1 


6 


6 


2 


1 


16 


Local Catalog 


12 


10 


7 




none 
reported 


31 


^hfirpfi Patrilna 


I Cm 




none 
reported 


none 
reported 


none 
reported 


1 7 


NUC 


1 


4 




1 1 


c 




cue, 


none 
reported 


1 


5 




4 


19 


Other 


none 
reported 


2 


none 
reborted 


3 




6 


TOTAL 


138 


103 


65 


41 




364 



Libraries using 1 or more finding tools - 138 
Libraries using 2 or more finding tools - 103 
Libraries using 3 or more finding tools - 65 
Libraries using 4 or more finding tools - 41 
Libraries using" 5 or more finding tools - 17 

* Among responding libraries CATALIST, 2nd Edition is the most frequently chosen 

lst> 2nd> and 3rd search order finding tool. 

• CATALIST is the most widely available finding tool in responding libraries. 
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These findings indicate a preference on the part of responding libraries to 
use local or regional finding tools before statewide or nationwide catalogs. 
System union catalogs, the catalogs of neighboring libraries, and shared cata- 
logs are the initial choice of over 59% of those libraries who have any choice 
at all. This figure contrasts sharply with that of responding libraries* choices 
for 2nd searches (see page 21) . The number of libraries preferring regional or 
local tools for second order searching declines to 32 (31%) and state or national 
union list preference increases to 67%. This agrees with what has long been 
known about ILL patterns - i.e., that librarians prefer to borrow from nearby 
libraries first and widen the geographic scope of their searches only when 
known local resources cannot provide the needed material. 

Questions 6, 7, and 11 addressed the ease of use of the finding tool and 
the adequacy of the instructional materials provided with the publication. Less 
than 2?i of those responding to questions 6 and 7 felt that additional instruc- 
tions or training were desirable. 98% rated CATALIST easy or very easy to use, 
less than 2% judged the finding tool difficult to use, and no one felt that the 
Second Edition was very difficult to use. 



Effectiveness of CATALIST, 2nd Edition 

Questions 8, 12, 13, and 14 of the survey were designed to provide data 
on the perceived effectiveness of the Second Edition and to identify problem 
areas for future improvement of the publication. The chart on page 23 displays 
a summary of the responses to question 8 which asked responding libraries to rate 
the effectiveness of CATALIST in support of various library functions (the same 
functions for which usage percentages were obtained from question 5). The ratings 
correlate very well with the pattern of usage revealed by the responses to ques- 
tion 5 (see page 19), with the higher effectiveness ratings assigned to those 
functions for which CATALIST is most often used. Only one of the specific uses, 
CATALIST as an aid in acquisitions decisions, measured in the survey received 
a lower than mid-point (5.0) score, though use of the finding tool as a cata- 
loging aid and as a public catalog were only marginally above the mid-point 
(5.1 and 5.2 respectively). 

The..ehart also lists the number of libraries who rated each specific usage 
and the percent responding with the lowest rating (1) and the highest (10) in 
each category of use, as well as the mode (most frequently reported) rating. From 
these figures additional inferences can be made with respect to the average effec- 
tiveness ratings. For example, the use of CATALIST for obtaining ILL holdings 
data received the highest overall effectiveness rating - 7.4. Very few (2, or 
}A%) of the 141 libraries rating this usage assigned the lowest rating and a 
relatively large number (31 or 21.9%) assigned it the highest rating. The most 
frequently assigned rating was 8 (out of a possible 10). This supports the con- 
clusion that most of the libraries responding to the survey have found CATALIST 
quite effective in support of ILL location finding. This contrasts sharply with 
the pattern evidenced in the usage receiving the lowest effectiveness rating - 
CATALIST used as an aid in making acquisitions decision - here the relatively 
large number of responding libraries who rated at the low end and the mode re- 
sponse of 2.0 indicate that even though the average rating of 4.2 is not much 
below the mid-range of 5.0, CATALIST has not been found very effective in support 
of this library activity. 



27 

•22- 




Effectiveness Ratings for Specific Uses of CATALIST, 2nd Edition 
(Scale = 1 to 10; 1 = very ineffective, 10 = very effective) 



Specific Use 


Number of Libraries 
Rating this Use 


Mode (Most Frequent) 
Rating 


Per Cent Rating 
Lowest (1) 


Per Cent Rating 
Highest (10) 


Average 
Rating 


Obtaining ILL 
holdings data 


141 


8.0 


1 AO/ 
1 A/o 




7 d 
/ * 4 


Bibliographic 
Verification 


115 


8.0 


4.6/0 


1 c Q<y 


D • ^ 


General Reference 


66 


5.0 


y . 1% 


/ . O/o 


R 7 


Soecific Subiect 
Reference 


87 


5.0 


5.7% 


O.U% 


b . o 


Aid for Aroui si ti ons 
Decisions 


19 


2.0 


15.8% 


b. j% 


A 0 


Cataloging Source 


6/ 


I .u 


1 0 . 7^ 




5.1 


Obtaining holdings 
data for direct loans 


62 


10.0 


6.5% 


16.1% 


6.5 


Public Use 


26 


1.0 


19.2% 


11.5% 


5.2 
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Question 12 was designed to measure the extent to which various potential 
problems with the finding tool have been perceived by users to be actual prob- 
lems. The chart on page 25 displays the responses to this question in a format 
similar to that used for the effectiveness ratings presented on page . 

The potential problem receiving thie highest (4.6) rating was that of "Did 
not list many titles for which we were searching." It will be remembered from 
the Contents Analysis section of this report that, at the time of the publication 
of the 2nd Edition, CATALIST had captured an estimated 12.2% (2.2 million) of the 
public library title location records potentially available in the state. The 
number of unique title records in the 2nd Edition was approximately 700,000. tt 
is probably impossible at this time to develop a reasonably accurate estimate of 
the number of unique titles collectively held by California public libraries; and 
even more difficult to estimate how well this statewide public library collection 
could function in terms of satisfying actual public demand as evidenced by ILL 
requests; nonetheless, the survey data supports the conclusion that users of the 
2nd Edition of CATALIST feel that its most significant problem is that not enough 
of the titles actually requested by patrons are found in the location tool. 

The two next most significant problems, as rated by the libraries responding 
to the survey, were that the holdings data listed proved inaccurate (4.4) and 
that of variations in entries, i.e., more than one entry for the same title (4.0). 
Both of these problems are inherent in the data base building, maintenance, and 
publication processes used to produce CATALIST (or any other large, batched, off- 
line database collecting data from many different sources). No machine algorithm 
has yet been devised that is capable of efficiently resolving differences in 
bibliographic data introduced by local cataloging variations; thus entry duplica- 
tion can only be eliminated by human intervention in the process. Inaccuracy in 
the holdings data may be introduced in a variety of ways, the most significant of 
which is the amount of time required for participating libraries to report with- 
drawals and for those reports to be processed for the next annual publication run. 
Nonetheless, the number of libraries rating these problems and the relatively 
high ratings assigned to them suggest that if CATALIST is to continue to be pub- 
lished as an annual batched offline product some additional effort should be ex- 
pended to minimize the occurance of duplicate entries and to speed the withdrawal 
process. \ 

Of the 148 libraires who responded to question 13 regarding the impact of 
CATALIST on ILL patterns 101 or 68% replied in the affirmative, i.e., judged that 
CATALIST, 2nd Edition had had a significant effect on ILL patterns. Of these only 
4 found that effect to be negative; one citing CATALIST's inaccuracy as a problem, 
one observing that the finding tool directed requests away from those libraries 
whose holdings were not listed, one who felt that CATALIST was not an adequate 
substitute for SCILL, and one who cited a 78% increase in requests from other 
libraries as having created ILL staffing problems. Most of the remaining 97 re- 
sponses cited improvements in the speed and accuracy of ILL procedures effected 
by doing bibliographic verification at the local level (19 responses), by going 
direct to the holding library rather than round robin or blind requests (13), and 
by lessened reliance on time consuming requests to the State Library for searching 
in the CUC (12). Six libraries reported that CATALIST had increased the level of 
ILL service offered to their patrons, either through an increased staff willing- 
ness to offer ILL. service or by providing patrons with direct access to the finding 
tool. One library reported that ILL volume had decreased due to the increased num- 
ber of direct loans resulting from CATALIST searching. The remaining responses 
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Specific Problem Ratings for CATALIST, 2nd Edition 
(Scale = 1 to 10; 1 = little or no problem, 10 = very serious problem) 



Problem 


Number of Libraries 
Rating this Problem 


Mode (Most Frequent)! 
Rating 


Per Cent Rating 
Lowest (1 ) 


Per Cent Rating 
Highest (10) 


Average 
Rating 


Titles Searched Not 
Listed 


117 


5.0 


8.5% : 6.8% 

J 


4.6 


Holdings Data 
Inaccurate 


88 


3.0 


•■! 

9.1% 


5.7% 


4.4 


Microfiche Format 
Troublesome 


31 


1.0 


25.8% 


0.0% 


3.2 


Entry Format Dif- 
ficult to Use 


53 


1.0 


28.3% 


1.9% 


2.9 


Variations in En- 
tries (Duplicate 
Titles) 


83 


5.0 


16.9% 


2.4% 


4.0 


Inaccurate Filing 
Order 


57 


3.0 


24.6% 


1.8% 


3.4 


Poor Visual Quality 
of Fiche 


33 


1.0 


36.4% 


0.0% 


2.6 


Difficult to Locate 
Correct Fiche 


33 


1.0 


45.5% 


0.0% 


2.6 
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either did not provide a statement of what the perceived impact had been ' . 
did not provide enough detail to categorize the response, A complete list of 
the responses to question 13 is contained in Appendix C. 



Question 14 was designed to elicit information on the perceived effective- 
ness of the subject section. 106 libraries responded to the question and rated 
subject reference use by staff as the most effective (6.0), followed by subject 
reference use by members of the public (4.5). The subject section was judged 
only minimally effect^ive for use as an aid in collection building (2.5) and in 
preparing bibliographies (2.6). 



User Preference for Publication Frequency and Format 

The responses to question 16 which was designed to measure the desirability 
of various publication options for future editions of CATALIST showed the least 
variation of any question in the survey. Fully cumulated annual editions re- 
ceived the highe^t^ rating (8.6 of a possib^le 10) with the lowest rating (3.3) 
assigned 'to the option of annual supplements. The option of producing a 6 month 
supplement as well as the fully cumulated annual edition received a relatively 
high rating of 7.6 reflecting the trade-off between decreased ease of use (two 
places to look versus one) and increased timeliness of information. 



Relative Value to Purchasers of CATALIST Publication Options 

Questions 17-21 (and, specific to the Subject Section, 15) of the survey 
were designed to provide some additional information on the desirability of the 
publication options listed in question 16 as well as some minimal "marketing" in- 
formation. It should be noted that nearly 93% of the survey respondents obtained 
their copies of the 2nd Edition through the LSCA-funded Finding List Project and 
therefore had not expended any local funds to acquire the finding tool. The Find- 
ing List Project was designed to encourage the use of CATALIST during its initial 
start-up publication years - until it had either proven its worth and was func- 
tioning (for publication costs) on a cost-recovery basis, on had been replaced by 
some other cost-recoverable access method. 

The reponses to question 17 through 21 are summarized in Appendix A and 
agree well with the desirability of the various publication options listed in 
question 16 (i-e., fewer responding libraries indicated a desire to purchase 
CATALIST at a lower cost for options that were judged less desirable in response 
to question 16). The chart on page 27 displays the number of responding libraries 
willing to purchase CATALIST for each of the publication options mentioned in the 
survey. 

The relative perceived value of the Subject Section of the 2nd Edtion may be 
estimated by the responses to question 15. 117 libraries responded, with an 
average value attached to the Subject of $29.50 - based on a price of $100 for 
the Author/Title section alone. Thus, for the 2nd Edtion, the Subject Section 
was judged to be worth about 30% of the costs of the Author/Title Section, and 
about 23% ,of the cost of the full set. 
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Comparative Value of CATALIST 
Publication Options 



CATALIST 
Publication Option 


Number of Libraries 
likely to purchase 
at 

lowest price ($150) 


Number of Libraries 
likely to purchase 
at 

$150 or more 


Desirability 

Rating 
(Question 16) 


Full cumulative annual 


72 


135 


8.6 


Full cumulative annual 
with 

6 month supplement 


60 


: 127 


7.6 


Cumulative Bienniel 
with 

3 6-month supplements 


57 


113 


4.5 


Cumulative Bienniel 
with 

1 annual supplement 


60 


109 


4.5 


Final Cumulative Base 

Edition with 
Annual Supplements 


64 


96 


3.3 : 
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Summary 



Data collected from over 170 (mostly public) libraries supports the conclu- 
sion that CATALIST, 2nd Edition, was the most widely available finding tool in 
responding libraries. This, by itself, is not particularly significant since 
only libraries known to have acquired the 2nd Edition were surveyed. Usage data 
indicate that the 2nd Edition was not only available in responding libraries, 
but was also receiving widespread and frequent use - on the average CATALIST 
was consulted on a daily basis (average use of 47 times per month per library) 
in responding libraries, with a conservatively estimated annual use of 95,000. 

The primary uses of the publication were in support of interlibrary and direct 
loan activities (almost 85% of the total usage), A significant number of libraries 
were making use of the 2nd Edition in support of reference activities and very 
few reported much usage for cataloging or acquisition purposes or for direct 
access by members of the public. Effectiveness ratings assigned to these various 
uses closely parallelled the amount of use for each activity with the higher (more 
effective) ratings received by those activities which accounted for the most use. 
CATALIST, 2nd Edition was judged by survey respondents to be quite effective for 
obtaining ILL holdings data, bibliographic verification, and obtaining holdings 
data for direct loans (average ratings of 7.4, 7.0, and 6.5 respectively, out of 
a possible 10) and of lesser effectiveness in support of reference activities 
(5.7-5.8), and of minimal effectiveness in support of other library functions. 

Responding libraries found lack of comprehensiveness (i.e., not enough titles 
listed) to be the most significant (4.6 out of a possible 10) problem for the 2nd 
Edition followed by inaccurate holdings data (4..4) and variations in entries (4.0). 
The first of these may be corrected automatically as the database grows; the 
second two probably cannot be eliminated in the present offline batched production 
process. No significant problems were identified with the microfiche medium it- 
self. Very few survey respondents had any trouble with the instructional materials 
or experienced difficulty in using the finding tool. 

If CATALIST is to continue to be published as a microfiche publication 
the most desirable publication frequency option was judged by responding libraries 

to be that of fully cumulative annual editions (continuation of the present oracticeU 
This preference is corroborated by the data gathered in response to the survey 
questions on the amount individual libraries would be willing to pay for the 
various publication possibilities, with 135 libraries indicating that they would 
pay $150 or more for the annual cumulative edition. 
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Hit Rate Survey Results 



The third and final segment of the CATALIST evaluation effort was a two 
week sample of ILL request searches and hits, A copy of the sampling instruc- 
tions and forms is provided as Appendix B of this report. The objective of 
the hit rate sample was to obtain data on one of the major uses of CATALIST 
(ILL location finding) in as wide and "real" a context as possible. Partici- 
pants in the hit rate survey were asked to change as little as possible in 
their methods and procedures for searching ILL requests* Separate search/hit 
rate tally sheets were kept for every finding tool in the participating 
libraries and searchers were requested to conduct their searches in exactly 
the same order that they normally would follow. As a consequence of this 
method of data collection the hit rate data presented in this report reflects 
tjie general or average experience of nearly 100 different libraries throughout 
the state - with all of the variations in staffing levels, searching expertise, 
and individual library and public library system resources coincident to a 
sample of that size. The data include the results of well over 16,000 searches 
conducted in late October or early November 1981* As in the general survey 
it v;as necessary to categorize certain groups of finding tools (System Union 
Catalogs, etc.) in order to organize the data in a manageable form. 

The chart on page 30 displays the results of the hit rate survey for al^ 
search orders. Delivery area and non-delivery area hits and rates are pre- 
sented separately (for the purpose^rAf the survey, "a delivery area hit" was 
defined as the obtaining of a holdiij^ statement for a library with which the 
responding ^library has a regular delivery arrangement). The 2nd Edition was 
searched a total of 4,139 times (nearly 25% of the total searches performed) 
during the sampling period, more than any other single finding tool (the 
System Union Catalogs total of 4,859 is composed of searches conducted on 
Black Gold's Union Catalog plus North Bay's Union Catalog plus ...). The 
overall hit rate for CATALIST, 2nd Edition was 6ir. (2,537 hits) comprised of 
a 29% delivery area hit rate and a 32% non-delivery area hit rate. Only two 
other finding tools or categories achieved a higher hit rate - OCLC and 
shared catalogs. The first of these is not surprising (hit rates in excess 
of 90% are frequently reported for OCLC's over 8 million title database) but 
the second, shared catalog, is significantly higher (76%) than would be pre- 
dicted for this type of finding tool which typically contains. the holdings 
of only 2 to 4 libraries. The most likely explanation for this is that com- 
bining the functions of a local and a union catalog produces a relatively 
large number of pre-located (100% hit rate) ILL requests. That is, since 
the shared catalog also functions as the local public access catalog, users 
consulting that catalog may .generate interlibrary loans that would not result 
if their access to the local collection did not include information on the 
holdings of other libraries. 
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Comparative Hit Rates by Finding Tool 



Finding Tool 


Total 


* 

Delivery Area 


Non-Delivery Area 


Total 


Searches 


Hits 


Rate 


Hits 


Rate 


Hits 


Rate 
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15% 


19 
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26 
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TOTALS 


16,656 


7,771 




2,297 


14% 


10,068 


61% 



• Category 6, shared catalogs, has on unexpectedly high hit rate {76%) for the size of 
such finding tools (typically 2 to 4 libraries' holdings)* A possible explanation is 
that when the local library's catalog includes the holdings of other libraries a higher 
number of requests for interloan service from an already known location may result. 

♦ Use of OCLC and RLIN for ILL searching is relatively low (less than S% of all searches). 
This is not unexpected since most libraries primary use of the bibliographic utilities 
is for technical processing. 
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The chart below is derived from the data displayed in the previous chart 
and supports the conclusion drawn from the general survey results - there is 
a marked preference in the libraries sampled for interlibrary loans from nearby 
sources. Thus system union catalogs where they exist are consulted more often 
than CATALIST or OCLC even though the overall hit rate experienced is lower. 
The benefits of borrowing from a neighboring library are preceived to be 
sufficiently high to offset the increased proportion of seaches that must 
be carried on to the next finding tool. 



Comparative Hit Rates By Scope of Finding Tool 
(Local/Statewide or National) 



Scope of. 
Finding Tool 


Total 
Searches 


Total 
Hits 


Hit 
Rate 


Local 
(2,5,6,9) 


10,829 


6,340 


59% 


Statewide/National 
(1,3,4,7,8) 


5,827 


3,728 


64% 


TOTALS 


16,656 


10,068 


61% 



The chart on page 32 displays the results of the same 16,000 searches by 
finding tool and search order. Search order is regarded as a significant 
variable in hit rate determinations because an unsuccessful search in any 
finding tool lowers the probability of finding the title listed anwyere. The 
degree to which this probability is reduced depends on the coinprehensiyeness 
of the finding lists previously consulted without success. The data displayed 
in the chart show clearly that several factors of which comprehensiveness of 
the tool is only one influence a searcher's decision of when to consult a 
particular finding tool. One of these factors is the increased value of a 
delivery area hit discussed above. This factor probably accounts for the data 
reflecting an overall 1st search hit rate of 61% versus a 64% 2nd search hit 
rate. [N.B. The low (43%) hit rate for 1st searches of system union catalogs 
probably reflects the fact that various round robin network searching shcemes 
were included in this category. Some of these reported very low hit rates. 
In addition a number of system union catalogs are in the initial states of 
development - these may be receiving 1st search use more as a means of identi- 
fying problems and developing experience with the new tool than as a means to 
maximize the efficiency of .local ILL operations.] Another factor is the 
searcher's own experience - both with particular finding tools and with 
requests for particular types of material. Many of the reporting libraries' 
searchers varied their search patterns considerably - probably on the basis 
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of the type of material requested. The only finding tool for which no 1st 
order searches were reported was RLIN (it should be noted that RLIN was 
experiencing difficulty in implementing a hardware/software upgrade during 
the period of the survey - this may have had an effect on individual libraries' 
willingness to devote scarce processing time to searching ILL requests and 
subsequently on the reported hit rate). These two variables - the experience 
and level of expertise of the searcher and the type of material requested 
were not measured in the survey but may have had a significant effect on any 
individual library's data. 



Hit Rates by Finding Tool /Search Order 



Finding Tool 


1st 
Search 


2nd 
Search 
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Search 


4th 
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11,047 
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1,092 
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1. A, single non-California user, after exhausting all in-state resources, 
uses CATALIST as its primary out-of-state finding tool and experienced 
a very high hit rate during the survey period. 

2. Probably reflects public access use of these tools. 



Sunmary 



While local libraries show a marked preference for local or regional 
interlibrary borrowing, and, therefore location searching, the more compre- 
hensive finding tool most often used after local or regional lists were con- 
sulted during the survey period was CATALIST (71% of all searches of state 
or national lists were CATALIST searches). The 2nd Edition experienced a 1st 
search order hit rate of 67% and an overall hit rate of 61% - ranking it third 
behind OCLC and shared catalogs (the 100% 1st search hit rate reported for the 
"other" category is anomalous - too few searches were reported to be considered 
significant). The high hit rate reported for shared catalogs is probably a 
reflection of a difference fri the way these tools are used (see dis- 
cussion above) rather than a measure of the comprehensiveness of such catalogs. 

The overall hit rate of 61% for CATALIST is composed of a 29% delivery 
area hit rate and a 32% non-delivery area hit rate - suggesting that, on the 
average, the chance of finding a location within a local library's delivery 
area is roughly equal to that of finding a location outside the region (of 
course, this is true only on the average - systems vary widely in terms of 
their representation in the 2nd Edition). 

CATALIST was searched for ILL locations a total of 4,139 times during the 
two week period - this equates to an annual search total of over 107,000 
searches for the 95 libraries who completed the hit rate survey. In its 
second year of publication CATALIST was searched for titles representing over 
37% of the ILL requests searched in responding libraries during the sampling 
period (4,138 + 11,047 initial searches). 
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Conclusions & Implications 



In its 2nd Edition CATALIST had achieved significant levels of usage and 
effectiveness in California libraries. By far the greatest amount of use 
was experienced in public libraries whose holdings also accounted for the 
greatest part of the information contained in the tool. Over 98% of the 
reported usage of CATALIST, 2nd Edition was in public libraries or public 
library systems and nearly 98% of its contents represent public library 
resources. 

Measured against one estimate of the total public library holdings in the 
State, the 2nd Edition contained slightly over U% (approximately 2.2 million 
holdings) of the whole. This 12% is distributed throughout the State in 
patterns that are more closely matched to the progress of individual libraries 
and systems towards conversion of their records to machine readable form than 
to the relative size and strength of library resources. Additional con- 
straints on the growth of the file include the availability of funding to add 
retrospective records and the degree of compatibility of locally produced 
machine readable records with the masterfile format. 

While 12% is a relatively small part of the total pub.lic library resources 
in the State, the 2nd Edition has performed well in support of public library 
resource sharing activities. As an ILL location finding tool users rated the 
publication at 7.4 on a scale of 1 to 10. As a bibliographic verification 
tool, it was rated 6.9 and for obtaining holdings for direct loan it was 
rated 6.5. An overall hit rate of 61% was reported for CATALIST, 2nd Edition 
in a two week sample of over 4,000 title searches in response to ILL requests. 
These figures suggest that considerably less than a full listing of all of 
the holdings of all public libraries in California is necessary to effectively 
contribute to resource sharing. 

This conclusion is substantiated by the responses of users when asked 
to conment on the effect of CATALIST, 2nd Edition on ILL patterns - all but 
4 of the comments were positive and spoke to increases in the speed and ease 
of both local activities (notably bibliographic verification) and network 
functions (more direct ILL's and lessened reliance on centralized manual union 
files such as the CUC). 

Relatively few CATALIST, 2nd Edition users reported significant problems 
in using the finding tool and, on the whole, did not rate those problem areas 
specifically mentioned in the survey as being very serious. On a scale of 
1 to 10 the highest rating (4.6) was reported for "Titles searched not listed 
pointing to a desire on the part of users for greater comprehensiveness in the 
finding toOl. The two next highest rated problem areas were - "holdings data 
proved inaccurate" (4.4) and "variations in entries" (4.0) illustrating some 
of the problems inherent in the building of larae bibliographic databases. 
While it is most likely that the major cause of inaccurate holdings data is 
the long "lag" time between the local library decision to withdraw a title 
and the actual removal of the holding statement from the union catalog, it is 
also true that attempts to minimize the number of variations in entries 
(duplicate titles) may contribute to the number of inaccurate holdings 
statements in. the file. *^ 
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Another "trade-off* inherent in the design of finding lists is that of 
comprehensi veness-versus local availability of the desired material. While 
date collected for the evaluation clearly support the desire of users for 
more titles in the finding tool, it is also clear that users will trade a 
higher hit rate (and, therefore, fewer multiple searches), to find a local 
source for the needed material. CATALIST may function nearly as well as 
their system union catalog or round robin searching in identifying local 
lending sources. 

Although most of the libraries using CATALIST, 2nd Edition received the finding 
tool at no cost to their jurisdictions through the Finding List Project, 
responses to the survey indicate that at least 135 of its users would be 
willing to purchase the publication at a price of $150 or more for an annual, 
fully cumulated, editjon. CLASS staff reports that the publication and dis- 
tribution costs of the 2nd Edition were $61 ,858 suggesting that about 1/3 of 
the publication and distribution costs of CATALIST would be recoverable from 
sales (maximum income was generated at the $150 pricing level). 
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Implications for the Future of CATALIST 
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The cost efficient provision of access to the monographic library re- 
sources of California is a complex process involving many steps. Verification 
and location finding are critical first elements in the design of efficient 
resource sharing systems but are by no means the deciding factors in determin- 
ing the final service delivery system design. Thus the future of CATALIST 
may ultimately be determined by other factors in the California library resource 
sharing environment than its cost and performance as a btbliographic verifica- 
tion and location finding tool. The implications di'scussed below, however, 
pertain solely to issues arising from the evaluation effort and the narrower 
focus resulting from concentration on the verification and location finding 
aspects of resource sharing. 

Size/Publication Medium 

While there was no indication from users that they found the 2nd 
Edition, with its 700,000 title listings and 2.2 million holdings statements, 
too cumbersome for convenient use it is clear that at some level well short 
of the estimated 18 million holdings statements representing California public 
libraries' monographic resources a microfiche access tool would cease to be 
practical given the current limits of affordable microform technology. It 
is also true that as the publication grows the variant entry and inaccurate 
holdings problems noted by users are likely to increase. 

Cost 

The cost for a single copy of the 2nd Edition (CLASS member price, without 
binders) was $160. Subsequent, larger editions would increase in price in 
proportion to the increase in size and in response to inflationary effects on 
the costs of microfiche production. While it does not appear, on the basis 
of the survey data, that the 2nd Edition could have recovered its publication 
and distribution costs from direct sales to users it is reasonable to assume 
that at least a greater portion of these costs could be recovered from sales 
in future years if the finding tool continues to prove as useful as users 
indicate and as its value grows with more, and more evenly distributed, title 
and holdings records. It is useful in this context to compare the cost of 
the microfiche publication medium with some gross estimates of the costs of 
alternatives. 

The 2nd Edition was comprised of approximately 340 fiche which equates 
to a cost of $.70/fiche (based on a publication run of 261 copies). About 
2,058 title entries appear on each fiche. If we assume that 20 title entries 
can be acconroodated on a page for print publication, 35,000 p.age_s would be 
required for one copy of the finding tool. For a small publication run of 
261 copies, the minimum a commercial printer would charge for typesetting, 
printing, and binding would be $.05/page. This would equate to a per copy 
cost of $1 ,750. 

To compare the costs of microfiche publication with online access is 
somewhat unfair since, presumably, online access could be designed to provide 
capabilities and functions beyond verification and location finding. At a 
minimum, however, online access would require that communication costs be 
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paid. A reasonable figure to assign to these costs would be in the range 
of $100 to $200 per month. At the mid-range figure of $150 this access 
would cost $1 ,800 annually. 

Local/ Regional /Statewide Access 

The results of the evaluation effort clearly substantiate the widely 
held notion that local libraries prefer to borrow from neighboring library 
jurisdictions. As a finding tool designed to serve the needs of California 
library users CATALIST naturally fared better in satisfying local and regional 
needs than other finding tools that include nationwide coverage. As the size 
of the statewide tool expands beyond its ability to prove useful, and, as the 
cost of alternative means of access decreases (two factors will serve to 
decrease the cost of online access: 1) the continuing reduction in the cost 
of data processing and telecommunications hardware; and 2) the continuing 
increase in the number of libraries implementing automated systems to serve 
other than resource sharing library purposes but which are compatible with 
online database access uses), there may well be a cross-over point at which 
it will make sense to distribute the database and implement one or more 
alternative access systems. 
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Appendix A 

CATALIST, Second Edition Summary of 
General Survey Responses 



1. Has your library received your copy of CATALIST, 2nd Edition: Yes □ No □ 



Number of Responses: 169 167 2 

2. Do you have a microfiche reader suitable for the use of . , „ 

CATALIST? YesD NoQ 

Number of Responses: 170 169 1 

3. Where is your library's copy of CATALIST, 2nd Edition, kept? 

Nut.^oer of Responses: 150 

a. Reference desk 50 

b. Technical service 12 

c. Circulation desk 10 

d. ILL dept. 56 

e. Other pubTic area (specify) 7 

f. Other non-public area (specify) 15 



4. Approximately how often per month is CATALIST, 2nd Edition, used in your 
library? 

Number of Responses: 168 

a. 0 times per month (Please see instructions in cover letter.) 

b. Fewer than 5 times per month 

c. 6-10 times per month 

d. 11-25 times per month 

e. 26-50 times per month 

f. 51-100 times per month 

g. 101-200 times per month 

h. More than 200 times per month 

5. Of your library's total use of-GATALIST, 2nd Edition please indicate 
below the approximate percentage of use for the following functions: 

Number of Responses: 152 



a. Obtaining ILL holdings data 53.8 % 

b. Bibliographic verification 2O.4 % 

c. General Reference Usage 

d. Specific Subject Reference Usage 5*9 % 

e. Acquisitions decisions 0.5 % 

f. Cataloging source ' 3.5 % 

g. Obtaining holdings data for direct loans ]0A % 

h. Use by members of the public 0,7 % 

i. Others; please specify 

0.5 % 

% 



TOOT 



13 
21 
33 
35 
27 
14 
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6. In your opinion, were the instructional materials provided 
with the fiche sufficient for obtaining the knowledge needed 

to use CATALIST effectively? Yes □ No □ 

Number of Responses: 152 150 2 

7. If no, what was needed in addition? 

Number of Responses: 2 

a. More detailed instructions? Ye*s [7] No Q 

b. Some on-site training? Yes fTl No! — | 

c. Other; please indicate: 0 



The following group of questions pertain to the effectiveness of CATALIST, 2nd 
Edition in performing or supporting various library functions. Please ask the 
person(s) on your staff who is in the best position to judge this effectiveness 
for eacn of the functions listed. 

8. How effective has CATALIST been for your library? On a scale of 1 - 10 
(10 being the highest), please rate how useful you have found CATALIST in 
each of the functions below. If you have found CATALIST equally effective 
for several functions, duplicate ratings may be assigned. 

Record a zero (0) if you^have not used CATALIST for a particular purpose. 

Number of Responses: 141 Average Effectiveness Rating 

a. Obtaining ILL holdings data 7.4 

b. Bibliographic verification 7.0 

c. General Reference Usage 5.7 

d. Specific Subject Reference Usage S.S 

e. Acquisitions decisions 4.2 

f. Cataloging source 5.1 

g. Obtaining holdings data for direct loans 6.5 

h. Use by members of the public 5.2 

i. Other functions: please specify and rate 



9. For sources for ILL holdings data for monographs please enter below, m the 
order you normally search , the finding tools your library owns and regularly 
uses (e.g., OCLC, RLIN, System Union Catalog, NUC, CATALIST, 2nd Edition, etc, 

Number of Responses: 143 Most Frequently Cited 

Search Order Finding Tool 



CATALIST 


[70 libraries) 


CATALIST 


[44 libraries^ 


CATALIST 


123 libraries) 



1 

2 

3 , , 

4 NUC (11 librariesT 



5 NUC (7 libraries) 
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lO*^: Approximately how many interlibrary loan requests does your library 
initiate monthly? 

Average 182 monograph requests/month 

Number of Responses: 138 

11. How easy to use is CATALIST, 2nd Edition? 

Number of Responses: 150 Number t)f Libraries 

a. Very easy to use — no problems. 81 

b. Easy to use — some problems. 67 

c. Difficult to use — significant problems. 2 

d. Very difficult to use ~ major problems. 0 

12. To what extent on a scale of 1 - 10 (10 being the greatest extent), were 
potential deficiencies to CATALIST a problem for you? Problems of equal 
magnitude may be assigned, equal ratings. 

Record a zero (0) if you did not find the characteristic listed a problem 
for you. 

Number of Responses: 117 Number of Average 

Libraries Rating 

a. Did not list many titles for which we were searching. ny 4 5 

b. Holdings data proved inaccurate, 88 4.4 

c. Microfiche form of catalog too troublesome 31 32 
for effective use. 

d. Format of entries difficult for use. 53 2.9 

e. Variations in entries, resulting in,; duplication of titles. 83 4.0 

f. Filing order inaccurate. 57 34 

g. Visual quality of fiche poor. 33 2 6 

h. Locating the correct fiche to search difficult. 33 2*6 



13. 



14, 



YesD HoQ 



In your opinion has the use of CATALIST, 2nd Edition had a 
significant impact on your library's/System's ILL patterns? 

Number of Responses: 148 101 47 

Please' indicate how usefiiV you have found the subject section of CATALIST, 
2nd Edition, in performing or supporting the following functions (use a 
scale of 1 - 10, 10 being the most useful): 

Number of Responses: 106 Average Rating 

a. As an aid in collection building 2.5 

b. As an aid in preparing bibliographies 2.6 
c* Subject reference use by staff 6.0 

d. Subject reference use by public 4,5 

e. Others; please specify and rate no response 
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15. What is the relative value of the subject section for your library? 
If the Author/Title section alone was priced at $100 please indicate 
In a range of 0 to $50 what you feel would be the appropriate price 
for the subject section. Average 

Number of Responses:' 117 29.5 



The following questions are designed to provide information useful in planning the 
form (cumulative vs. supplement) and frequency of future editions of CATALIST. As 
CATALIST grows in size the cost of producing the list increases also. The frequency 
and comprehensiveness of each edition also have a significant impact on the production 
cost of the tool. We are therefore asking you to answer the following questions based 
on your best judgement of what your library would purchase at the prices indicated. 
Please ask the person(s) on your staff who actually makes such decisions to complete 
questions 17 - 21 below . 



16. If cost were not a consideration, please rate on a scale of 1 - 10 
(10 being the most desirable) the desirability from your library's 
point of view of the following publication options for CATALIST: 

Number of Responses: 130 _ Average Rating 

a. Full cumulative annual edition (present practice) 8.6 

b. Full cumulative annual edition with six months supplement 7.6 

c. Full cumulative biennual edition with 3 six month supplements 4.5 

d. Full cumulative biennual edition with 1 annual supplement 4.5 

e. Full cumulative base edition with annual supplements thereafter 3.3 



17. Assuming CATALIST were to continue to b.e published as a full cumulative annual 
edition, please indicate how many copies your library would be likely to 
purchase at the prices indicated: 

Number of Responses: 135 

We would purchase one 



First Copy copy at (check highest 
Price applicable) 



$150 72 libraries 

$200 16 libraries 

$250 24 libraries 

$300 11 libraries 

$350 4 libraries 

$400 2 libraries 

$450 1 library 

$500 4 libraries 

$600 0 libraries 

$700 0 libraries 

$800 1 library 
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18. Assuming CATALIST were to be published as a fully cumulated annual edition with a 
six (iionth supplement, please indicate how many copies your library would be 
likely to purchase at the prices indicated (prices ar^ annual cost per copyj. 

Number of Responses: 127 • 



First Copy 
Pri ce 


copy at (check highest 
applicable) 


$150 


fin 


1 1 Kv^A PC 
1 1 Ui a 1 1 Co 


$200 


11 


libraries 


$250 


23 


libraries 


$300 


9 


libraries 


$350 


5 


libraries 


$400 


3 


libraries 


$450 


0 


libraries 


$500 


5 


libraries 


$600 


0 


libraries 


$700 


0 


libraries 


$800 


0 


libraries 



19. If CATALIST were published as a full edition every two years with 3 six month 
supplements, please indicate how many copies your library would be likely to 
purchase at the prices indicated (prices are annual cost per copy): 

Number of Responses: 113 



First Copy 
Price 

$150 
$200 
$250 
$300 
$350 
$400 

\m 

$600 
$700 
$800 



We would purchase one 
copy at (check highest 
applicable) 
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12_ 



16 



1 



libraries 

libraries 

libraries 

libraries 

libraries 

library 

library 

libraries 

libraries 
libraries 
libraries 
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20, Assuming CATALIST were to be published as a full edition every two years with 
an annual supplement, please indicate how many copies your library would be 
likely to purchase at the prices indicated (prices are annual cost per copy): 

Number of Responses: 109: 

We would purchase one 



First Copy 
Price 


copy at (check highest 
applicable) 


$150 


60 


1 i brari es 


$200 


12 


libraries 


$250 


17 


libraries 


$300 


8 


libraries 


$350 


- 2 


libraries 


$400 


3 


libraries 


$450 


0 


libraries 


$500 


6 


libraries 


$600 


0 


libraries 


$700 


0 


libraries 


$800 


1 


library 



21. Assuming CATALIST were to be published in one final fully cumulated edition 
\ with annual supplements thereafter, please indicate how many copies your 
-library would be likely to purchase at the prices Indicated (prices are 
annual cost per copy) : 

Number of Responses: 96 



We would purchase one 



First Copy copy at (check highest 
Price applicable) 



$150 64 libraries 

$200 10 libraries 

$250 11 libraries 

$300 4 libraries 

$350 1 library 

$400 1 library 

$450 1 library 

$500 3 libraries 

$600 0 library 

$700 1_ library 

$800 Q_ library 
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Appendix B 



CATALIST Evaluation Search/Hit Rate Data Collection form 



Instructions: We are trying to get a picture of where CATALIST fits in relation to 
the other major finding tools in use throughout the state. To accomplish this we 
must gather data not only on CATALIST hits but also on all other major finding 
tools in regular use. Therefore please complete one form for each finding tool your 
library regularly searches to obtain location information for ILL. If your library 
forwards all ILL requests to a centralized searching facility DO NOT COMPLETE THESE 
FORMS. If your library does its own searching for ILL locations: 

1. Complete the lines for library name and finding tool name (one form for 
each finding tool your library regularly uses). 

2. Place each form in a convenient, prominent location near the appropriate 
finding tool (OGLC terminal, NUC shelves, CATALIST microfiche reader, etc.) 

3. Other than tallying searches and results, do not vary your library's normal 
ILL searching strategies and procedures. If the results of the sample suggest that 
your library might want to modify your current practice we would like to hear about 
it, but conduct "business as usual" during the sample period. 

4. Instruct all staff who regularly search for ILL locations to tally each 
search and its results on the form provided in the following manner: 

5. Tally marks for searches and hits are to be entered in 
the row that indicates the "order searched" for the individual 
title being searched, e.g. if the title being searched in OCLC 
has previously been searched without success in the System 
Union Catalog, the tally marks for search and hit should be 
placed in the second row since this is the second finding 

tool in which this title has been searched. (See sample 
search) 

6. Hits are to be recorded in two categories: "Delivery 
Area" and "Other". For the purposes of this survey a "Delivery 
Area Hit" is a library with which your li.brary has a regular 
delivery arrangement. All other hits are "Other". 

7. Record only one hit per search, even if the finding 
tool indicates multiple locations. If both "Delivery Area" 
and "Other" hits result record only the "Delivery Area" hit. 

8. We are requesting a two week sample period. Enter the beginning date and 
ending date for your library's sample in the space provided at the bottom of each 
form. If, at the end of the sample period, ILL staff feel that the sample is in 
some way atypical or non-representative of normal activity please include a brief 
statement indicating what was unusual and any possible reasons for the atypical ity 
of the data. 

9. When the sample period is over please total the tally marks in each row/ 
-column*"box" (unless you needed multiple forms for a single finding tool you can 
just write the arable numeral total in each- box - we don't need a "clean' form); 
add up the 'Delivery Area' and 'Other' hits and enter in the "Total Hits" column; 
and enter the total of each column in the "TOTALS" row across the bottom. 
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10. Return all completed forms to: 

Cameron Robertson 
Systems & Automation Office 
California State Library 
P.O. Box 2037 
Sacramento, CA 95809 

no later than November 16, 1981 . 



n. Some cases (for example. Round Robin TWX lists; system union catalogs 
that also serve as local catalogs, etc.) will require special arrangements to 
ensure uniform data collection. If your system or library uses such tools or if 
you have any questions about the survey in general or the Search/Hit Rate Data 
Collection forms please call Cameron Robertson (916) 322-0365. 

12. A sample search is attached to illustrate the above instructions. 



S ample Search 

An ABC public library patron has requested Three Men In A Boat by Jerome K. Jerome. 

The title has been verified (this verification search is not counted for the survey 
unless the verification tool used doubles as a location fTnJing tool). The ABC PL 
is an OCLC cataloging and ILL participant and the searcher decides to try "online" 
first. The online search reveals 4 locations for the title, all within the state - 
but none of which are on the XYZ system's delivery route. The searcher records the 
search results on the survey form next to the OCLC terminal: (Note that even though 
four locations were found only 1 "hit" is recorded.) 



CATALIST Evaluation Search/Hit Data 

Library Keportinp: 
Findins Tool: 



Order 
Searched 


Number of 
Searches 


Delivery Area 
Hits 


Other 
Hits 


Total 
Hits 


First time this 
Title searched 


1 




1 




Second tipe this 
^ ^^^c^ed 






r 
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The searcher would prefer to find a location within the library's delivery area. 
The ABC public library participates in the XYZ cooperative library system which 
publishes a COM system union catalog, and the ILL searcher decides to look there. 
The union catalog search indicates that a system member holds a copy of the title. 
The searcher records the result of the search on the survey form next to the micro- 
fiche reader: 



CATALIST Evaluation Search/Hit Data 

Library Reporting: /I ^ C ^vV\ »'(L 

F?ndins Tool: 



Order 
Searched 


Number of 
Searches 


Delivery Area 
Hits 


Other 
Hits 


Total 
Hits 


First time this 
Title searched 










Second time this 
Title searched 


1 


1 






iMxiL-Lilje t his 

















Note that the tally marks appear in the second row since this is the second time 
this title has been searched. 

The searcher feels that with an in-system location and 4 "back-ups" there is an 
excellent chance to obtain a copy of the title for the patron and continues the 
ILL process without searching further. 

Even though CATALIST was never searched in this example we still need the data 
for the two finding tools that were consulted. Please tally each search of each 
finding tool with the sole exception of those titles which require more than five 
searches. For these record the results of the first five searches only. 

N.B. If your library uses more than one person to search a single title you will 
need to devise a means of letting an individual searcher know where they are in 
the search order. If the search order is rigid (RLIN always first; CATALIST always 
second, ...) then you can simply cross out the non-occurring boxes on the collection 
form. If the order varies from title to title a pencilled record of sources searched 
on the original P-slip or request form will allow a searcher to enter the tally marks 
in the appropriate row. 
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Library Reporting: 
Finding Tool: 



Order 
Searched 


Number of 
Searches 


Delivery Area 
Hits 


Other 
Hits 


Total 
Hits 


First tirae this 
iitxe searcned 










Second time this 










"^Third time this 










Fourth time this 
Title searched 










Fifth time this 
Title searched 










TOTALS 











Start Date: 



End Date: 



Library Reporting: 
Finding Tool: 



CATALIST Evaluation Search/Hit Data 
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Order 
Searched 


Number of 
Searches 


Delivery Area 
Hits 


Other 
Hits 


Total 
Hits 


First time this 
Title searched 










Second time this 
Title searched 










Third time this 
Title searched 










Fourth time this 
Title searched 










Fifth time this 
Title searched 










TOTALS 

■ ■ ■ * 




-^T 







Date: 
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End Date: 



Appendix C 

Comments in response to Question 13: '*In your opinion has the use of CATALIST, 
2nd Edition had a significant impact on your library's/System's ILL patterns? 
If yes, please state briefly what the impact has been." 

1. More ILL'S. 

2. Expediates loans, generally there have been no problems. 

3. System Research Center sends requests directly to libraries showing 
holdings in CATALIST making ILL process faster. Bibliographic 
verification eliminates errors. 

4. Because we do not own the major tools that give holdings, it gives us a 
quick way to check for holdings, particularly of technical and "non-public 
library type" materials. 

5. Speed, accuracy ~ gives us some idea of where, outside the MCLS system 
the material we are searching for can be found - our only fource for this. 

6. Allows us to pinpoint locations for rush requests; gives an additional 
verification source. 

7. High proportion of hits has meant we offer ILL service to patrons, or 
simply try CATALIST automatically if unable to supply from within the 
system. 

8. We have gone direptly to locations listed on CATALIST, with a definite 
improvement in turnaround time instead of using our NSCLS ILL Network 
(which did not provide locations information in any form). 

9. It has been a great help in both verification and locating titles for 
specific subject reference. 

10. We can now order materials direct from system members which hold them, 
instead of going through Stockton. 

11. The CATALIST has cut the search time for our library by at least 50% 
in verifying requests. 

12. For the first time we are receiving 80% of Interlibrary loansl 

13. The loss of SCILL has had a large impact on the library staff time for 
ILL has tripled. Can't satisfy as many reouests. No longer have access 
to universitites. CATALIST does not fill the vaccuum created by SCILL's 
loss. 

14. We are now able to go directly to owning library on ILL thus relieving 
some of the burden on system personnel . 

15. We are able to list for our ILL center locations of materials outside 
system. 
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16 • Sending more direct requests » rather than going through a system. 

17. Irrmediate knowledge of books available to borrow for our patr'^ns. Also 
more use of our books through ILL requests to us. 

18. Helps in specific location of books and verification of bibliographic 
information. 

19 • We've had more requests for our books. 

20. The advent of CATALIST has changed the order of procedures for billing 
interlibrary loan requests. 

21. We find that in researching a book, it is a lot easier to go to the CATALIST 
than searching through several reference books. Public as well as staff 
use CATALIST for bibliographic verification and identification of materials 
oh a particular subject. 

22. Immensely has improved our ability to locate ILL items, in time and money. 

23. Quicker turn around time by eliminating CSL Union Catalog check - so we 
don't go to CSL if found in CATALIST. 

24. With demise of SCILL and cutback on services provided, by MCLS and now 
Santiago Library System, we rely heavily on CATALIST for locations. 

25. Involves more search time, since we formerly had only BIP and System fiche 
for tools. However* patron requests are being filled faster than before. 

26. More out-of-system borrowing. This is offset by difficulty in obta.ning 
material . 

27. It has meant that we can verify more of our own requests, instead of having 
to send to Eureka for verifidation. Also, it seems our requests are being 
processed a little faster because we are able to provide EIC with initial 
location information. ] 

28. Eliminated the approximately two week wait for locations from State Library 
on about 50-60% of requests previously sent to State Library for locations. 

29. Simplification of the verification process. 

30. It has largely replaced CSL Union file as source of state holdings, parti- 
cularly for books published in 60's and 70*s. 

31. In-house use faster than union checks at headquarters. 

32. Users are very enthusiastic about it, and now that they know what books 
are available they are requesting many more books than previously. 

33. It is better than nothing but is too inaccurate. Helps often in verifying 
materials but not good as ILL tool. 

34. Saves time; we go to CSL less often. 
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35. Provides backup to System Union Catalog. 

36. Less use of State Library List. 

37. Instead of sending requests to System ILL headquarters, we use CATALIST 
and request loans directly, after checking CLSI county holdings. 

38. Helps to find locations in California. 

39. Makes verification easier and more often able to verify at this level. 

40. Prior to obtaining CATALIST, 2nd ed. we had no proper verification tool 
or union catalog. Monographic ILL consisted of shooting in the dark. It 
has meant a 99.9% improvement. 

41. We check in CATALIST before sending requests to State Library for locations. 
Saves much time. 

42. We send more requests to individual California libraries rather than de- 
pending upon the California State Library to provide us with locations. 

43. With the re-organization of our reference center (LOCNET) we have started 
using CATALIST heavily. 

44. ILL requests/deliveries exact and easier. Teachers (academic and vocational 
education, counselors, therapists, etc.) Law Research reference tool... 

45. As verification tool it speeds up inter-library loan. 

46. All titles are verified through CATALIST first. If locations are found, 
we only ask for a PLS Union Catalog check, otherwise we ask PLS to go on 
to CSL for listings. 

47. We send fewer requests to State Library now, thereby shortening fill time. 

48. In house, ILL librarian goes directly to CATALIST to search for verification 
and holdings. 

49. North Bay holdings have been included in the 2nd Edition and this has been 
a considerable help in locating listings of more recent titles. 

50. Tended to direct requests toward those libraries whose holdings were listed 
and therefore away from some of the other libraries in the system. 

51. It has been valuable as a verification tool thereby speeding up the ILL 
process. 

52. Often will obviate the need to ILL patron, in finding that a local 
library has the book, will travel to that library to get it. So if any- 
thing, it has cause ILL's to decrease somewhat. 

53. Requests to borrow our material increased over 78%! We received 257 re- 
quests one week! .5 staff time and all reimbursements to the system. 

54. Much easier verification to facilitate rapid ILL service. 
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55. Makes verifying a request easier and often speeds up the amount of time 
it takes to obtain a title by requesting direct. 

56. R has provided the only "union catalog" access we have to other North 
State libraries without going to the State Library. It therefore shortens 
our response time for those titles which are listed. 

57. Increases response time; provides only subject access for system reference 
questions other than BIP. Most ILL requests have already been searched in 
CATALIST before they come to us, so we don't actually use CATALIST for ILL 
as much as individual libraries. 

58. Some increased ILL usage because of public use. 

59. Provides a starting point from which to initiate a search for a specific 
monograph not in our collection. 

60. It is now possible to pinpoint a holding and order directly from most 
responsive source. 

61. Gave us more areas (libraries) to search. 

62. It has decreased the staff time needed to verify ILL requests. 

63. Except for esoteric requests, CATALIST because of its accessibility, 
ease and speed of use, including its acceptability in serving as biblio- 
graphic verification, is our first source for ILL outside our system. It 
has had a major impact on the speed and costs of serving a significant 
portion of our ILl requests. Needs improvement, but I hate to think of 
trying to handle ILL's without CATALIST or similar successor. 

64. To reduce the need for TWXing the State Library for sources. Their re- 
sponse time averaged two weeks. A time saver. 

65. Processing ILLs is significantly faster. 

66. Saves time in determining holdings. 

67. There has been an increase in the number of ILL requests initiated by the 
library. 

68. Other library holdings are more readily available. 

69. By ordering directly from holding library instead of requesting a search 
at State Library. 

70. It provides holding information where CSL leaves off. 

71. We are loaning to more out of system libraries and borrowing more from out 
of system sources. 

72. Information regarding holdings in libraries for ILL loans. 

73. It helps sometimes in shortening ILL process by being able to go directly 
to a holding library. 
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74. It hasn't changed our ILL patterns but we have been able to verify and 
correct many ILL requests. Books on esoteric subjects have been found. 

75. Faster bibliographic verification. 

76. We start with CATALIST when searching for locations of requested titles. 
Find it goes a lot faster than checking other bibliographic sources and 
sending the request on to State Library, which was our pattern in the past, 
(Pre CATALIST) 

77. We are aware immediately of some other library's holdings. 

78. Does save a few phone calls, 

79. Easily accessible source for ILL searches. 

80. Only for verification. 

81. It is helping to locate books outside our immediate system and request 
these materials either directly (ala, whisper-writer) or through sis/ 
locnet . 

82. Both in sending and receiving ILL requests, volume has greatly increased; 
this is a burden as well as an advantage because it overloads our ILL per- 
son - our system (NBC) used to do much of this work. 

83. Some impact. We are doing a few more ILL's outside our system. 

84. CATALIST is used in place of Union Catalog in times prior to OCLC conversion. 

85. Eliminates in many cases the necessity to ask State Library for holdings. 
Turn around time has thus improved. 

86. We have been able to eliminate about 25^0 of our sending to CSL for locators. 
We gain access to 1979-80 titles which are not available through CSL. 

87. Increased awareness of holdings in local and regional libraries 

88. Patrons receive items sooner - not necessary to wait for reply from Cali- 
fornia State Library. 

89. Several items I would have turned to academic library for were located 
close by in public library. 

90. Listings search usually successful with CATALIST II, enabling us to quickly 
send request on TWX. 

91. We can send requests directly to libraries in our system (now that our TWX 
has been removed and there's no 'Vound robin") 

92. CATALIST has given us the capability of going directly to the probable 
source and has eliminated the intervening step of querying the CSL. 

93. ProvTded another source and more titles are available. 
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94. Use CATALIST as first checking after our own catalog. 

95. It saves time, as about eight out of ten titles are there. 



GO 
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